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'  During  the  period  of  the  grant  Professors  David  Heath,  John  f  » 

Muckstadt,  and  Carol  Shilepsky  have  conducted  research  on  the  inter-  m  m 

changeability/substitutability  problem  for  recoverable  items  (items  * 

Pj 

that  are  subject  to  repair  when  they  fail).  This  problem  arises  when 
recoverable  items  can  be  substituted  for  one  another  during  the  repair 


of  an  assembly. 
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The  long  range  objective  of  the  research  project  is  to  develop 
a  method  that  the  Air  Force  can  use  to  assist  in  the  management  of 
interchangeable  recoverable  items.  To  accomplish  this  goal  we  have 
proposed  to  analyze  several  simplified  problems  that  will  give  us 
insight  into  the  form  of  the  optimal  or  near-optimal  policy  for  the 
real  problem.  Specifically,  we  proposed: 

(a)  to  understand  fully  the  behavior  of  a  single-echelon, 

two-item  system  over  an  infinite  horizon  when  the  failure 
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processes  for  the  items  are  independent,  stationary  Poisson 
processes  and  the  repair  times  are  exponential; 

Cb)  to  develop  methods  for  finding  optimal  and  near-optimal 
policies  for  the  situation  described  in  (a); 

(c)  to  extend  the  results  for  topics  (a)  and  (b)  to  systems 
having  many  items; 

(d)  to  extend  the  analysis  to  situations  in  which  there  are  two 
echelons  (depot-base  structure)  and  many  items  where,  as 
before,  the  failure  processes  for  the  items  are  independent, 
stationary  Poisson  processes  and  repair  times  are  exponential; 

(e)  to  develop  methods  for  finding  the  optimal  or  near-optimal 
operating  policy  for  the  situation  described  in  (d); 
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During  the  period  of  the  grairrifatre"*  In  Lei'cliaiigbobility /-substitutability 
problem  for  recoverable  items^was  studied.  The  objective  of  the  grant  was 
to  develop  methods  for  managing  interchangeable  recoverable  items.  Research 
c  :,t<rted  under- -ttie-grant)  has  led  to  a  full  understanding  of  the  behavior 
of  i  single-echelon,  two- item  system  over  an  infinite  horizon  when  the' 
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-syst,™  have  been  found.  ;  These  nethods  have  teen  extended  to  methods  for 

°rtinv3i  ^oli-cies  for  managing  the  single-echelon' 
system  with  several  interchangeable  recoverable  items. 


(f)  to  study  the  problem  when  failure  and  repair  processes\  . 

_ 

need  not  be  stationary  and  the  time  horizon  is  finite  foi\  vV 

V'1*' 

both  single-  and  two-echelon  systems;  and 


(g)  to  develop  heuristic  dispatching  rules  for  the  dynamic  ' 

environment  described  in  (f). 

To  date  we  have  completed  objects  (a)  and  (b),  developing  heuristics 
to  find  a  near-optimal  policy  for  the  single-echelon,  two-item  system. 

We  have  almost  completed  objective  (c),  developing  two  heuristics  which 
should  produce  a  near-optimal  policy  for  the  system  having  many  items. 

We  have  begun  working  towards  objectives  (d)  and  (e),  establishing  the 
framework  and  the  procedure  which  will  be  followed  in  analyzing  the 
two-echelon  system.  Details  of  the  progress  made  towards  reaching  these 
objectives  are  given  below. 

In  meeting  objectives  (a)  and  (b),  we  studied  the  interchangeability/ 


substitutability  problem  for  two  items  that  fail  at  a  single  location. 

The  failure  processes  for  these  items  are  assumed  to  be  independent, 
stationary  Poisson  processes ,  and  repair  times  are  exponentially  distri¬ 
buted.  The  system  studied  is  assumed  to  be  a  closed  system;  that  is, 
no  items  are  added  to  or  deleted  from  the  system.  Based  on  these 
assumptions  we  first  showed  that  the  problem  could  be  viewed  as  a  Markovian 
decision  problem  for  which  there  exists  a  stationary  optimal  policy.  Since 
our  main  goal  is  to  find  methods  that  can  be  used  to  solve  the  real 
problem,  we  next  developed  various  approaches  for  finding  optimal  and 
near-optimal  policies. 

The  first  approach  we  took  was  to  formulate  the  decision  problem  as 

a  linear  programming  problem.  The  number  of  states  in  the  Markovian 
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decision  problem  is  so  large  that  the  linear  programming  method  would 
not  be  a  practical  method  for  finding  optimal  policies.  We  developed 
a  procedure  for  avoiding  this  difficulty.  We  found  that  the  behavior 
of  a  system  with  many  parts  could  be  satisfactorily  studied  by  con¬ 
sidering  a  smaller  system  with  fewer  parts,  and  thus  fewer  states  in 
its  associated  Markovian  decision  problem.  The  number  of  states  can  be 
further  reduced  by  eliminating  certain  unlikely  states  from  consideration. 
With  fewer  states,  it  becomes  practical  to  use  linear  programming  to 
obtain  an  optimal  solution  for  the  smaller  system.  This  solution  can 
be  translated  into  a  near-optimal  solution  for  the  original  system. 

Thus  we  can  use  linear  programming  to  obtain  near-optimal  solutions  for 
the  single-location,  two-item  system. 

Next,  we  studied  the  solutions  obtained  by  linear  programming  and 
were  able  to  identify  some  properties  that  an  optimal  policy  should  possess. 
We  developed  a  simulation  approach  that  exploits  these  properties  that  an 
optimal  policy  is  conjectured  to  have.  This  heuristic  method  is  compu¬ 
tationally  efficient  and  finds,  at  least  for  the  cases  tested,  a  nearly 
optimal  policy.  (The  details  of  the  results  of  this  work  can  be  found 
in  the  attached  technical  report,  entitled  "An  Analysis  of  a  Single 
Location  Inventory  Problem  for  Two  Interchangeable  Recoverable  Items.") 

In  working  towards  objective  (c),  we  extended  the  system  studied 
in  (a)  and  (b)  to  include  several  interchangeable  recoverable  items. 

We  found  that  the  number  of  states  and  possible  actions  grew  so  huge 
that  any  attempt  to  use  linear  programming  to  find  an  optimal  policy 
would  be  impractical.  Hence  we  turned  our  attention  to  the  development 
of  heuristic  methods  which  should  produce  a  good  policy. 
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Our  first  approach  involved  regarding  the  multiple-item  system  as 
a  sequence  of  two-item  systems.  At  each  stage  in  the  sequence  the 
linear  programming  procedure  developed  for  objectives  (a)  and  (b)  is 
used  to  find  a  near-optimal  policy  for  a  segment  of  the  multiple-item 
system.  This  method  produces  a  rule  prescribing  when  a  substitution  is 
to  be  made.  (The  details  involved  in  calculating  and  implementing 
this  substitution  rule  can  be  found  in  the  attached  Technical  Report, 
entitled  "A  Procedure  for  Finding  a  Nearly  Optimal  Policy  for  the 
Inventory  System  with  Several  Interchangeable  Recoverable  Items.") 

Our  second  approach  involved  formulating  the  multiple-item  decision 
problem  as  a  policy  improvement  problem.  Beginning  with  the  policy  of 
never  making  any  substitutions,  the  special  structure  of  this  problem 
can  be  exploited  to  permit  us  to  carry  out  the  policy  improvement  algo¬ 
rithm  for  a  single  iteration,  producing  an  improved  substitution  policy. 
Preliminary  computational  experience  indicates  that  this  policy  is  a  good 
one. 

In  beginning  to  work  towards  objectives  (d)  and  (e),  we  have 
established  the  framework  for  studying  the  two-echelon  system.  We  are 
constructing  a  flexible  simulation  model  that  will  enable  us  to  evaluate 
and  compare  the  performance  of  several  alternative  depot  and  base  deci¬ 
sion  rules.  This  simulation  will  also  be  used  to  evaluate  the  performance 
of  the  two  heuristic  rules  developed  under  objective  (c). 

We  have  presented  the  results  of  our  research  at  the  International 
Meeting  of  the  Institute  of  Management  Sciences  in  Hawaii  in  June  1979, 
at  the  Multi-Level  Inventory  Conference  in  Philadelphia  in  October  1979, 
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and  at  the  Annual  Meeting  of  the  Operations  Research  Society  of  America 
in  Washington,  D.C.,  in  May  1980.  Furthermore,  we  plan  to  prepare  a 
paper  having  the  same  title  as  the  first  attached  technical  report  and 
submit  it  for  publication  in  the  Naval  Research  Logistics  Quarterly. 

Throughout  our  investigations  we  have  been  in  contact  with  Lt.  Col. 
Jon  Hobbs  and  Mr.  Victor  Presutti  of  Headquarters,  Air  Force  Logistics 
Command  (AFLC/XRS).  Their  guidance  and  assistance  in  providing  experi¬ 
mental  data  has  been  extremely  valuable. 
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ABSTRACT 


In  this  paper  we  examine  the  interchangeability/substitutability 
problem  for  two  recoverable  items  that  fail  at  a  single  location.  We 
assume  the  failure  processes  for  each  type  of  item  are  independent, 
stationary  Poisson  processes.  We  also  assume  the  repair  times  are 
exponentially  distributed.  Furthermore,  we  assume  that  the  system  is 
a  closed  system,  that  is,  no  items  ire  added  to  or  deleted  from  the 
system.  We  first  consider  a  discrete-time  problem  and  show  that  this 
problem  is  a  Markovian  decision  problem.  We  then  show  that  for  this 
problem  there  exist  optimal  stationary  Markov  control  policies.  Next 
we  formulate  a  continuous  time  model  and  show  how  to  find  the  optimal 
stationary  Markov  control  policy  using  linear  programming.  Unfortunately, 
this  approach  is  impractical  for  solving  most  real  problems.  Consequently 
we  have  established  and  explored  some  of  the  properties  that  we  feel 
an  optimal  policy  should  possess.  A  discussion  of  these  properties  is 
given  in  Section  IV.  Lastly,  we  will  describe  a  heuristic  that  can  be 
used  to  find  a  good  policy.  This  method  is  an  efficient  simulation  search 
method  that  finds  policies  having  the  properties  we  conjecture  an 
optimal  policy  should  possess. 


I .  INTRODUCTION 


During  the  past  15  years  a  substantial  amount  of  research  has  been 
conducted  related  to  the  management  of  recoverable  items,  that  is,  items 
subject  to  repair  when  they  fail  t1.2'3.4-5.6-7-8.^^.12^3-!4],  A 
number  of  mathematical  models  have  been  developed  that  can  be  used  to 
determine  optimal  stockage  levels  for  each  recoverable  item  in  both  single 
and  multi-echelon  systems.  Most  of  the  models  are  based  on  the  assumption 
that  the  items  are  independent.  That  is,  the  failure  processes  among  the 
items  are  assumed  to  be  independent.  Some  recent  research  has  been  devoted 
to  dependencies  in  the  demand  process  by  recognizing  that  certain  recoverable 
items  have  a  hierarchical  design  ^ 1 , - .6 ,13 ,14] ^  ,  jr  -these  items,  the 
failure  of  a  recoverable  component  results  in  a  demand  for  both  a  spare 
component  and  the  assembly  containing  the  component.  However,  in  all  of 
the  models  presented  to  date  the  replacement  rule  for  a  failed  component  is 
the  same:  replace  all  failed  units  with  a  serviceable  spare  item  of  the 


same  type. 


In  this  paper  we  examine  a  problem  that  arises  when  items  are  some¬ 
times  interchangeable  or  can  be  substituted  for  one  another  during  repair, 
frequently  it  is  possible  to  repair  a  broken  assembly  using  several  differ¬ 
ent  types  of  parts;  however,  choosing  the  "correct"  part  to  use  to  repair 
the  assembly  is  not  based  on  engineering  considerations  alone.  Using  one 
type  of  item  to  complete  the  repair  rather  than  using  a  second  type  of  item 
can  cause  subsequent  parts  shortages  that  can  be  avoided.  This  can  occur 
because  some  items  are  "more  useful”  than  others.  For  example,  suppose 
there  are  only  two  types  of  items  in  the  system.  The  "more  useful"  item 
can  be  used  to  satisfy  a  demand  for  either  type  of  item  whereas  the 
"less  useful"  item  can  only  be  used  to  satisfy  demands  for  its  own  type 
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of  item.  Many  such  interchangeable/substitutable  items  are  found  in  the 
Air  Force.  This  is  particularly  the  case  for  electronic  items.  In  some 
instances,  newly  designed  items  can  be  used  to  replace  older  units  when  they 
fail;  however,  these  older  units  cannot  be  used  to  repair  a  newer  generation 
of  an  assembly. 

In  this  paper  we  will  examine  the  interchangeability/substitutability 
problem  for  two  items  that  fail  at  a  single  location.  We  assume  the  failure 
processes  for  each  type  of  item  are  independent,  stationary  Poisson  processes. 
We  also  assume  the  repair  times  are  exponentially  distributed.  Furthermore, 
we  assume  that  the  system  is  a  closed  system,  that  is,  no  items  are  added 
to  or  deleted  from  the  system.  We  first  consider  a  discrete-time  problem 
and  show  that  this  problem  is  a  Markovian  decision  problem.  We  then  show 
that  for  this  problem  there  exist  optimal  stationary  Markov  control  policies. 
Next  we  formulate  a  continuous -time  model  and  show  how  to  find  the  optimal 
stationary  Markov  control  policy  using  linear  programming.  Unfortunately, 
this  approach  is  impractical  for  solving  most  real  problems.  Consequently 
we  have  established  and  explored  some  of  the  properties  that  we  feel  an 
optimal  policy  should  possess.  A  discussion  of  these  properties  is  given 
in  Section  IV.  Lastly,  we  will  describe  a  heuristic  that  can  be  used  to 
find  a  good  policy.  This  method  is  an  efficient  simulation  search  method 
that  finds  policies  having  the  properties  we  conjecture  an  optimal  policy 
should  possess. 


II.  A  TWO-ITEM  SINGLE  LOCATION  SYSTEM:  . 
A  DISCRETE  TIME  MODEL 


In  this  section  we  present  a  discrete-time  model  for  the  inter¬ 
changeability/substitutability  problem.  We  examine  this  discrete-time 
model  since  it  yields  a  particular  form  for  the  optimal  control  policy. 

In  particular,  we  show  that  for  the  associated  Markov  decision  problem 
there  is  a  stationary  Markov  control  policy  which  achieves  the  lowest 
average  back-order  level.  This  result  provides  the  motivation  for 
restricting  attention  to  Markov  control  policies  in  the  continuous  time 
model  developed  in  the  next  section. 

To  simplify  the  analysis  and  discussion  we  restrict  our  attention 
to  a  single  location  system  with  only  two  types  of  items:  type  1  and 
type  2.  The  assemblies  in  which  these  items  are  installed  are  called  units ; 
we  also  assume  that  there  are  two  types  of  units  and  that  each  unit  contains 
only  one  item  of  the  types  considered.  Furthermore ,  when  a  type  1  unit  or  a 
type  2  unit  fails,  we  assume  that  it  can  be  repaired  with  a  serviceable 
type  2  item.  Type  1  items  can  also  be  used  to  repair  failed  type  1  units: 
however,  type  1  items  cannot  be  used  to  repair  type  2  units.  For  example, 
two  units  might  be  different  "generations"  of  a  computer  found  in  a  fire 
control  system;  the  items  might  be  old  and  new  versions  of  an  integrated 
circuit  board  found  in  the  computer.  The  newer  version  of  the  circuit  board 
can  be  used  in  both  generations  of  the  fire  control  system  computer;  but, 
the  old  generation  circuit  board  is  incompatible  with  the  newer  fire  control 
system  computer. 

Let  N.  be  the  number  of  units  of  type  i  and  M.  be  the  number 

i  l 

of  spare  items  of  type  i  in  the  system.  Thus  there  are  a  total  of 

N.  +  M.  items  in  the  system.  Let  n..  be  the  number  of  tvpe  i  items 
i  i  '  n  -  . 


installed  in  type  j  units,  and  m;  be  the  number  of  servicable  type  i 
items  in  spare  stock. 


Note  that  according  to  the  substitution  rules  we  have  established 


n^0  is  always  zero.  Thus  the  five  numbers  m^  ,  m^  ,  n^  ,  n0^  ,  and 

n.„  specify  the  disposition  of  all  items.  The  number  of  tvoe  i  items 
z  z. 

1 

in  repair  is  given  by  N.  +  M.  -  (m.  +  E  n..)  ,  and  fh'e  number  of  back- 

11  1  j  =  l  1]  2 

orders  associated  with  type  j  units  is  N.  -  E  n.^  . 


We  assume  the  system  operates  as  follows  in  the  discrete  time  model. 

At  each  time  t  =  n  •  (At),  n  =  0,  1,  ...  certain  actions  are  available. 
These  actions  correspond  to  installing  some  items  currently  in  spare  stock 
in  appropriate  units  lacking  an  item.  After  installation  of  items, 
failures  may  occur.  We  presume  that  items  fail  independently  of  one  ar.cthe 
and  that  each  item  cf  type  i  installed  in  a  unit  of  tvpe  j  has  proba¬ 
bility  A„  •  (At)  of  failing  (where  (At)  is  small  enough  so  that  these 
numbers  do  not  exceed  1).  Next,  items  which  have  failed  are  removed  from 
the  units  and  sent  to  repair.  Items  are  repaired  independently;  we  presume 
that  each  item  is  repaired  during  this  time  interval  with  probability 
r .  •  ( At )  . 

l 

After  this  sequence  of  action-failure-repair  we  begin  again  at  the  new 
time  (n+1)  •  At  by  selecting  another  action.  The  system  continues  to 
operate  in  this  manner  for  an  indefinite  length  of  time. 

The  number  of  backorders  during  the  action-failure-repair  cycle  is 
defined  to  be  the  number  of  backorders  which  exist  immediately  after  the 
action  (unit  repair)  is  taken  and  before  the  failures  occur. 

We  wish  to  choose  these  actions  which  minimize  the  average  number 


of  backorders. 
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The  selection  of  the  particular  sequencing  of  events  -  action,  failure 
repair  -  was  not  made  arbitrarily.  This  sequence  was  selected  so  that  the 
problem  could  be  formulated  as  a  Markovian  decision  problem  (average  cost 
model).  (A  discussion  of  Markovian  decision  problems  can  be  found  in 
reference  10.) 

Notice  that  for  any  policy  (that  is,  a  specification  of  the  actions  to 
be  taken  for  all  possible  situations)  there  is  positive  probability  (actually 
bounded  away  from  zero)  that  after  one  cycle  of  action-failure-repair  all 
items  which  were  in  use  will  have  failed  and  been  repaired.  Hence  there 
is  a  state  (namely  that  with  no  items  installed  and  all  in  spare  stock) 
for  which  every  action  taken  at  every  state  gets  to  that  state  in  one  step 
with  probability  greater  than  or  equal  to  >  0  .  By  Ross  [10]  (Theorem 
6.17,  Corollary  6.20,  and  the  remarks  following  Corollary  6.20)  there  is 
then  an  optimal  stationary  Markov  control  policy.  This  policy  can  be  com¬ 
puted  by  a  technique  involving  linear  programming;  this  technique  is  adapted 
to  the  continuous-time  model  developed  in  Section  III. 


6 


III.  A  TWO-ITEM  SINGLE  LOCATION  SYSTEM: 
A  CONTINUOUS  TIME  MODEL 


We  now  consider  the  continuous  tine  model  corresponding  to  that  of 
Section  II.  The  notation  describing  th=  numbers  of  units  and  items  and 
the  state  of  the  system  remain  the  same.  ' 

We  now  suppose  that  the  failure  times  of  type  i  items  installed  in 
type  j  units,  are  independent  exponentially-distributed  random  variables 


with  mean  1/A.^  ,  and  that  the  repair  times  are  independent,  exponential 
random  variables  with  mean  1/r^  •  As  before,  the  measure  of  performance 


of  the  system  is  the  average  number  of  backorders.  Motivated  by  the 


results  of  the  previous  section  we  shall  consider  only  stationary  Markov 


control  policies. 


Since  it  would  seem  unreasonable  to  allow  backorders  for  tyre  1  units 


if  there  are  type  i  items  in  spare  stock  we  consider  the  installation  of 
type  i  items  in  type  i  units  to  be  automatic  and  not  subject  to  control. 

Thus  the  actions  available  involve  only  the  installation  of  type  2  items 
in  type  1  units.  We  shall  compute  the  optimal  stationary  Markov  control 
policy  which  takes  action  only  when  the  system  changes  state  due  to  an  item 
failing  in  service  or  being  returned  from  repair.  Finally,  we  allow  only  the 
substitution  of  one  type  2  item  into  type  1  units  at  a  time. 

When  the  process  jumps  to  a  new  state  there  are  (possibly)  two  actions 
available:  do  nothing,  or  install  a  type  2  item  in  a  type  1  unit.  Of  course, 
if  there  are  no  type  2  items  available  or  no  backorders  associated  with 
type  1  units,  then  there  is  only  one  action  available:  do  nothing. 

Following  Ross  [10]  we  allow  randomized  actions;  thus  corresponding  to 

al  a2 

state  S  there  are  two  numbers  P  and  Pg‘  (non-negative  and  summing  to  1) 
giving  the  probabilities  of  selecting  action  a^  ( =  do  nothing)  or  a^ 

(=  put  one  type  2  item  in  a  type  1  unit).  These  P's  completely  specify  the 
control  policy. 


Once  these  P's  are  specified  the  process  which  results  is  a  stationary 


Markov  process.  In  fact,  if  we  consider  the  process  which  specifies  the  state 
the  system  most  recently  jumped  to  and  the  action  taken  there,  this  process  is 
a  Markov  chain  and  we  can  compute  its  stationary  transition  probabilities 
and  then  the  average  cost. 

Let  "g  be  the  equilibrium  (or  stationary)  probability  that  this  process 

most  recently  jumped  to  state  S  and  that  the  action  taken  there  was  a  .  Th°' 

we  must  have  E  E  a!?  Z^  =  E  zt  where  a_  (S,^S„)  is  the  rate  at 
a  S2  S1S2  S2  a  S1  S1S2  1  * 

which  transitions  to  state  will  occur  if  the  current  state  is  S2  and 

the  action  taken  is  "a"  ,  and  =  -E  so  that  the  column-sums 

b2b2  S  blb2 

s2) 

of  the  a-matrix  are  zero.  (The  subscripts  may  seem  reversed  in  the  above. 

This  is  because  in  the  usual  Markov  chain  matrix  notation  the  order  of 
vector-matrix  multiplication  is  the  reverse  of  that  used  in  the  usual  L.P. 
notation,  which  we  adopt  here.) 

The  cost  associated  with  each  state,  C(S,a)  ,  is  the  number  of  back¬ 
orders  (total  for  units  of  type  1  and  type  2)  associated  with  state  S  if 
action  a  is  chosen. 


We  wish  to  minimize  the  cost-per-unit-time  given  by  E  E  Z  C(S,a) 

S  a  S 

subject  to  the  equilibrium  equations  EE  ac  c  Zq  =  E  Z^  and  to  the 

a  S  Si  2  b2  a  bl 

condition  E  E  =  1 
S  a  5 

-l  >•  ■ 


(Mote  that  in  some  states  substitution  obviously  cannot  be  performed; 
for  these  states  we  can  simply  ignore  (that  is  set  to  zero-)  the  corresponding 
zf  for  a  =  SUBSTITUTE) . 
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I  1  ? 


The  solution  to  the  above  stated  L.P.  provides  the  values  for  the  P's 
(and  hence  the  policy)  as  follows: 

Pg  =  P{take  action  a  when  in  state  S} 

zs 


whenever  E  2^  /  0  .  When  E  2^  =  0  ,  state  i  is  never  reached  bv  the 
b  b  b  b 

controlled  system  and  thus  it  makes  sense  to  leave  the  action  chosen  there 
undefined . 

Suppose  T  represents  the  number  of  realizable  states  in  the  system 

(a  state  is  realizable  whenever  E  2^  >  0  ).  Then  the  above  optimization 

a 

problem  has  T  equality  constraints  and  T  basic  variables  (note  there  is 

one  redundant  constraint).  If  state  S  is  realizable,  then  >  0  for  at 

least  one  action  a  .  Consequently  at  least  one  2^  is  positive  for 
S=l,  T  .  This  implies  that  for  each  state  S,  can  be  positive  for 

only  one  action  a  .  Thus  must  be  either  0  or  1  . 

A  program  was  written  to  generate  the  input  for  a  linear  programming 
package  (MPSX).  The  output  was  then  submitted  to  MPSX  and  (in  almost  all 
cases)  an  optimal  strategy  was  obtained. 

To  present  the  results  of  the  computations  we  notice  that  under  our 
control  policies  the  number  of  variables  necessary  to  describe  the  state  of 
the  system  can  be  reduced  from  the  five  above  to  three  as  follows:  Since 

we  never  allow  backorders  for  tvpe  i  units  if  there  is  serviceable  spare 

.  2 

stock  on  hand;  *V.  -  E  n..  and  m.  cannot  both  be  positive.  We  thus  set 

i  .  .  1 1  i  r 

l  =  i 

2 

s.  =  m.  -  [N.  -  E  n..J  ,  which  is  the  net  inventory  of  items  of  type  i 

i  1  l  .=.  ]i 

(it  in  negative  if  there  are  backorders  associated  with  type  i  units). 


i 

! 


The  variables  ,  s2  and  n^  then  describe  the  state  of  the  system. 

Finally,  since  at  each  state  the  control  policy  merely  specifies 
"substitute"  or  "don't  substitute"  it  suffices  to  graph  the  set  of  states  S 
at  which  one  would  perform  a  substitution.  Graphs  of  the  optimal  "substitution 
set"  S  for  several  situations  are  given  in  Figure  1,  Figure  2,  and  Figure  3. 


E( Backorders) 


1 

n2.l 


Substitute 
Don't  Substitute 


Example  A.  Optimal  Policy 
Figure  1. 
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IV.  THE  FORM  OF  AN  OPTIMAL  POLICY 


To  enable  us  to  find  approximations  to  an  optimal  policy,  we  examined 
the  structure  of  the  set  of  states ,  S  ,  from  which  a  type  2  item  should 
be  substituted  into  a  type  1  unit.  Clearly 

S  -  {(s^  s2>  n21)  :  s1  <  0,  s2  >  0}  . 


Every  such  subset  gives  rise  to  a  control  policy;  however,  there  are 
three  properties  (monotonicity  relationships)  that  one  might  expect  the 
optimal  subset  (SQ  )  to  have.  Willingness  to  perform  a  substitution 
depends  on  the  number  of  type  1  units  out  of  service,  the  number  of  type  2 
items  in  spare  stock,  and  the  number  cf  type  2  items  already  installed  in  type 
1  units.  Intuitively,  for  fixed  s2  and  n2^  ,  as  s^  decreases  there  is 

at  least  as  great  a  need  to  substitute  type  2  parts  in  the  type  1  units. 

We  express  this  as 

MR1)  (sr  s2,  n21)  £  SQpt  ,  s  <  sx  ,  implies  (s,  n^)  £  SQpt  . 

Similarly,  a  greater  supply  of  type  2  spares  should  imply  an  equal  or  greater 
willingness  to  substitute.  This  property  is  expressed  as 


MR2)  (s1,  s2,  n21)  £  SQpt  ,  s  >  s2  ,  implies  (s^  s,  n21)  e  Sopt 


Finally , 


MP3 )  (Sj,  s0,  n21)  £  SQpt  ,  n  <  n01  ,  implies  (s^  s?,  n)  £  SQpt  . 

Since  S  is  a  set  in  three-dimensional  space  we  can  draw  its  graph  in 
sections;  thus  if  we  let  each  section  correspond  to  a  fixed  value  of  s^  we 
would  expect  the  graph  to  look  like  that  of  Figure  4. 

In  each  optimal  strategy  computed  using  the  linear  programming  method 
described  in  Section  III  the  properties  MR1)-MR3)  held.  On  the  basis  of 
this  conjectured  form  of  the  optimal  policy  we  developed  a  procedure  to 
examine  states  for  possible  inclusion  in  S  . 


c» 


V.  APPROXIMATIONS  TO  AN  OPTIMAL  POLICY 


To  find  a  good  approximation  to  the  optimal  substitution  policy  we 
employed  a  search  technique  which  proceeds  as  follows:  begin  with  S 
empty  (i.e.,  with  a  policy  which  allows  no  substitution);  repeatedly 
consider  adding  one  point  at  a  time  to  S  by  comparing  the  performance 
of  the  system  using  the  augmented  S  with  the  current  policy;  then  add 
the  point  if  the  level  of  performance  is  higher. 

The  first  states  considered  are  those  for  which  s  =  -1  .  In  this 
set,  the  most  likely  candidate  for  membership  in  S  is  (-1,  M  ,  0); 
i.e.,  allow  substitution  when  there  is  a  type  1  unit  out  of  service  only 
if  there  are  M,,  type  2  items  in  stock  and  none  already  JLpstalled  in 

i. 

type  1  units.  If  this  state  is  added  to  S  ,  s2  is  successively  decreased 
by  one  unit  as  long  as  the  inclusion  of  (-1,  s2,  0)  improves  the  policy. 

Then  n^^  is  incremented  by  i  and  (-1,  M^-l,  1)  is  considered  (note, 
if  n„,  =  1  ,  then  at  most  Mj-i  serviceabie  spare  type  2  items  can  be  in 
stc-k).  Again,  sn  is  decreased  until  there  is  no  further  policy  improvement. 
We  then  increment  n^  again.  After  all  appropriate  states  of  the  form 
(-1,  3.,  n„,  )  have  been  added  to  S  ,  those  for  which  s.  =  -2  are  con- 

2  Zx.  1 

sidered  in  a  similar  fashion.  We  take  advantage  of  property  MR1)  to  include 
automatically  in  S  each  state  (-2,  s?,  n0^)  such  that  (-1,  n2i^  ^as 
already  been  added  to  3  .  This  significantly  reduces  the  number  of  comparisons 
which  have  to  be  made . 

An  alternate  search  pattern  was  considered  in  which  (-N^,  Mj ,  0 )  is 
the  first  state  examined  for  inclusion  in  3  .  In  this  state  all  type  1  units 
are  out  of  service  and  all  type  2  items  are  in  spare  stock.  Intuitively, 
this  is  the  state  from  which  one  would  be  most  likely  to  allow  substitution 
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and  hence  an  appropriate  starting  point  for  the  search.  The  disadvantage 
of  this  approach  was  that  MRi)  could  not  be  used  to  increase  the  computational 
efficiency,  and  hence  the  first  method  was  used. 

The  search  technique  involved  being  able  to  compare  the  performance  of 
two  policies.  We  developed  two  methods  for  this:  one,  analytic,  which  gave 
an  exact  number  for  the  expected  backorders  under  a  given  policy,  and  the 
second,  simulation. 

In  the  analytic  method  we  observe  that  the  substitution  rules  fcr  a  given 
policy  and  the  transition  probabilities  depend  only  on  the  present  state, 
hence  the  system  is  a  Markov  chain.  The  method  consists  of  generating  the 
steady  state  equations,  finding  the  equilibrium  probability  distribution 
for  the  chain  and  calculating  the  expected  number  of  backorders  under  the 
equilibrium  distribution.  This  number  can  be  compared  to  the  expected  back¬ 
orders  for  the  same  policy  augmented  by  one  state  as  required  by  the  search 
procedure . 

A  serious  disadvantage  of  this  method  of  policy  comparison  is  related 
to  the  number  of  states  in  the  chain  and  hence  the  number  of  equations  to  be 
solved  for  the  equilibrium  distribution.  The  number  of  states  for  a  system 
with  M.  units  of  type  i  and  M.  items  of  type  i  is 

(».  +  M.  +  1)(M„  +  1)(K„  +  M„/2  +  1)  . 

11  l  4.  t. 

For  example,  a  relatively  small  system  with  =  10  and  =  5  has  1248 

states.  When  the  number  of  units  and  items  grows  by  a  factor  of  n  ,  the 

3 

number  of  states  increases  roughly  by  a  multiple  of  n  .  The  time  required 

to  solve  the  equations  corresponding  to  the  enlarged  system  then  increases 

9 

oy  approximately  a  multiple  of  n  .  We  are,  therefore,  critically  limited 
in  the  size  of  the  system  we  can  investigate  using  the  Markov  analysis  to 
compare  policies. 
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The  major  advantage  of  this  approach  is  that  the  answers  are  exact  and 
hence  twc  policies  can  be  compared.  The  similar  results  obtained  by  linear 
programming  and  by  the  search  technique  with  analytic  policy  comparison 
suggested  that  the  search  mechanism  is  valid  and  encouraged  the  development 
of  a  more  efficient  method  of  policy  comparison. 

The  simulation  method,  which  provides  the  only  tool  suitable  for  the 
analysis  of  large  systems,  performs  two  simulations,  one  for  each  policy 
and  uses  the  state  at  which  the  policies  differ  as  a  starting  point  for  the 
simulation.  In  one,  the  substitution  of  a  type  2  part  is  made,  and  in  the 
other  it  is  not  made.  The  two  simulations  are  then  run  until  they  both 
reach  the  same  state.  This  state  is  not  necessarily  the  one  in  which  the 
simulations  started.  If  either  system  reaches  the  initial  state  in  which 
the  desirability  of  substitution  is  being  questioned,  the  substitution  is 
net  made. 

'■■/hen  the  two  systems  reach  the  same  state  the  run  is  terminated  and  the 
difference  between  the  number  of  backorder-days  is  recorded;  call  the  result 
(say  the  number  of  backorder-days  with  the  extra  substitution  minus  the  back¬ 
order-days  without  it)  for  run  i  3^  . 

We  wish  to  determine  E(E^)  ,  since  if  E(B^)  >0  we  should  not  perform 

the  additional  substitution  while,  if  E(B,  )  <  0  we  should.  To  estimate 
n  n  2 

E(3.)  we  computed  Z  B.  and  Z  Bt  ;  we  could  then  estimate  E(3  )  and 
"  i=l  1  i=l  1  i 

V(Bl)  and  construct  confidence  intervals  for  E(B^)  .  Large  groups  of  runs 

were  made;  after  each  group  of  runs  a  confidence  interval  was  computed  at  a 

selected  confidence  level.  If  this  confidence  interval  did  not  include  the 

origin  the  procedure  was  terminated  and  the  appropriate  strategy  was  selected 

as  optimal.  Moreover,  if  the  confidence  interval  did  include  zero  but  was 

shorter  than  some  pre-selected  tolerance  level,  the  procedure  was  terminated 

and  it  was  concluded  that  both  policies  gave  nearly  the  same  performance  level. 
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(.Actually  in  our  runs  this  outcome  did  not  occur.)  It  should  be  noted  that 
although  we  were  using  a  sequential  procedure  we  used  analysis  appropriate 
for  a  single  sample  and  that  this  is  not  precisely  correct.  However,  in 
each  case  the  results  obtained  by  simulation  were  the  same  as  those  obtained 
by  analytic  comparison.  Moreover,  the  simulation  was  computationally  superior 
in  two  respects:  the  amount  or  computer  time  did  not  grow  as  rapidly  with 
increased  system  size  as  it  did  for  the  analytic  method,  and,  perhaps  even 
more  relevant  in  terms  of  absolute  limitations,  the  simulations  did  not 
require  the  iaree  amounts  of  storage  necessitated  by  solving  such  large 
systems  of  equations.  Substitution  sets  (S)  generated  by  the  search  pro¬ 
cedure  for  the  same  examples  presented  in  Section  III  are  found  in  Figures 
5 ,  6  ,  and  7 . 

In  conclusion,  we  make  the  following  observations: 

1)  the  exact  solutions  obtained  through  linear  programming 
and  the  approximate  solutions  obtained  with  the  search 
support  the  assumption  that  the  form  of  the  optimal  policy 
satisfies  MR1)  -  MR3); 

2)  comparison  of  exact  results  and  those  obtained  by  a  search 
with  analytic  policy  evaluation  indicate  that  a  search 

is  an  effective  method  of  policy  improvement; 

3)  the  correlation  between  results  of  search  with  analytic 
comparison  and  simulation  comparison  indicates  that  the 
simulation  method  is  valid  and  hence  gives  us  a  method 
for  finding  an  approximation  to  an  optimal  policy  which 
can  be  applied  to  large  systems. 
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VI.  A  SUMMARY  AMD  COMMENTS  CONCERNING  FUTURE  EFFORTS 

While  we  now  have  a  method  for  policy  approximation  which  can  be  used 
on  large  systems,  there  are  several  possibilities  for  further  work  with  this 
model.  Continued  investigation  of  the  structure  of  an  optimal  policy  (i.e.. 
is  it  linear  in  any  of  its  variables?)  might  suggest  a  reduction  in  the 
number  of  states  to  be  considered  for  inclusion  in  S  during  a  search  for 
an  approximation  to  the  optimal  policy.  A  more  precise  comparison  of  the 
performance  of  the  exact  and  approximate  solutions  should  be  made  to  find 
a  balance  between  reduced  backorders  and  computational  accessibility. 

In  addition,  we  plan  to  use  the  insights  obtained  through  the  study 
of  this  highly  simplified  inventory  model  as  a  basis  for  our  future  efforts 
to  study  more  complex  situations. 

Unfortunately,  many  real-world  considerations  are  not  addressed  in  our 
simplified  model.  For  example,  we  have  ignored  the  fact  that  a)  sometimes 
a  family  of  substitutable  items  may  consist  of  more  than  two  items,  b)  items 
are  normally  stocked  at  more  than  one  location,  c)  the  failure  and  repair 
distributions  may  not  be  stationary,  d)  the  planning  horizon  may  be  of  such 
a  short  duration  that  infinite  horizon  models  may  be  inappropriate,  and 
e)  the  system  stock  level  for  each  item  may  not  remain  constant  for  an 
extended  period  of  time.  We  plan  to  address  many  of  these  issues  in  our 
future  work. 
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1.  Introduction 


The  effective  management  of  recoverable  spare  stock  leads  to  many 
important  problems  in  inventory  theory  (See  [1]  for  references  to  earlier 
work).  Very  little  work  has  been  done  in  the  case  that  recoverable  items 
are  interchangeable  or  that  one  can  be  substituted  for  another  during 
repair.  Such  a  problem  can  arise  when  one  part  is  more  useful  than  another 
and  can  be  used  to  satisfy  a  demand  for  itself  or  for  another  type  part, 
while  the  less  useful  type  part  can  only  be  used  to  satisfy  a  demand  for 
itself.  A  substitution  (i.e.,  of  the  more  useful  part  for  the  less  use¬ 
ful  part)  may  lead  to  subsequent  shortages  of  the  more  useful  type  part, 
and,  although  it  will  prevent  an  immediate  backorder,  may  not  always 
be  the  best  action.  An  inventory  policy  would  specify  conditions  under 
which  a  substitution  should  be  made. 

In  Cl]  Heath,  Muckstadt  and  S'nilepsky  consider  the  following  inter¬ 
changeability/substitutability  problem  for  two  items  that  fail  at  a  single 
location.  The  failure  processes  for  each  type  of  item  are  independent, 
stationary  Poisson  processes.  Failed  items  are  repaired  and  the  repair 
times  are  exponentially  distributed.  Furthermore,  the  system  is  closed, 
that  is,  no  items  are  added  to  or  removed  from  the  system. 

They  first  consider  a  discrete  time  formulation  of  this  model  and 
show  that  it  is  a  Markovian  decision  problem  for  which  there  exist  optimal 
stationary  Markov  control  policies.  They  next  formulate  a  continuous 
time  model  and  show  how  to  find  the  optimal  stationary  control  policy  using 
linear  programming.  Because  of  the  size  of  most  real  problems,  the  linear 
programming  approach  cannot  be  applied;  however,  its  application  to  small 
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problems  led  to  conjectures  concerning  properties  that  an  optimal  solu¬ 
tion  should  possess  [Heath,  Muckstadt  and  Shilepsky].  Based  on  these 
properties  an  efficient  simulation  search  method  was  developed  to  find 
good,  but  not  optimal,  policies. 

In  this  paper  we  discuss  computational  considerations  relating  to  the 
linear  programming  solution  of  small  problems.  We  first  examine  a  tech¬ 
nique  for  improving  the  efficiency  of  the  linear  programming  solution  by 
providing  a  good  starting  basis.  Second  we  discuss  techniques  for  redu¬ 
cing  the  number  of  variables  in  the  linear  programming  problem.  These 
techniques  enable  us  to  solve  problems  which  had  previously  been  inaccess¬ 
ible  either  because  of  their  size  of  numerical  problems. 

In  [2]  Heath,  Muckstadt  and  Shilepsky  present  a  scaling  technique  by 
which  one  may  approximate  a  large  problem  by  a  small  one,  find  the  optimal 
solution  for  the  latter  and  use  it  to  find  a  solution  to  the  large  problem. 
The  effectiveness  of  the  scaling  technique  depends  heavily  on  the  ability 
to  solve  small  problems  efficiently  and  accurately. 
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II .  Formulation  of  the  Model  and  Solution  by  Linear  Programming 

We  consider  the  following  model.  We  suppose  that  there  are  two 
types  of  items  in  the  system:  type  1  and  type  2.  The  assemblies  in  which 
these  items  are  installed  are  called  units;  we  also  assume  that  there  are 
two  types  of  units  and  each  unit  contains  only  one  item  of  the  types  con¬ 
sidered.  All  items  are  assumed  to  be  stored  at  a  single  location. 

We  assume  that  items  of  type  2  can  be  placed  in  units  of  type  1  or 
type  2,  but  that  items  of  type  1  can  be  placed  only  in  units  of  type  1. 
Finally,  we  assume  that  times  to  failure  and  repair  times  are  exponentially 
distributed. 

For  the  model  we  suppose  that: 

1)  There  are  units  of  type  i  and  JL  +  M.  items  of  type  i  are 

available  (i.e.,  there  are  J'L  spare  items  of  type  i), 

2)  The  state  of  the  system  at  any  instant  can  be  specified  by  the  five 
numbers: 

n..,  the  number  of  type  i  items  installed  in  type  j  units, 

I] 

i  >i- 

tru,  the  number  of  serviceable  type  i  units  in  spare  stock. 

The  number  of  type  i  items  in  repair  is  then  given  by  -  (iru  + 

and  the  number  of  back-orders  associated  with  type  j  units  is 
Nj  -  \  n  „ .  We  suppose  that  the  failure  times  of  items  of  type  i  in¬ 
stalled  in  units  of  type  j  are  independent  exponential  random  variables 
with  mean  1/X.4,  and  that  repair  times  are  independent,  exponential  random 

^  J 

variables  with  mean  1/r. *  The  measure  of  performance  of  the  system  under  any 
substitution  policy  is  the  expected  number  of  backorders  when  the  system  is 
in  equilibrium. 


ii  r — 
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Under  any  reasonable  control  policy  (i.e.,  rule  specifying  which  items 
to  use  to  repair  various  units  under  all  possible  circumstances)  the  num¬ 
ber  of  variables  necessary  to  describe  the  state  of  the  system  can  be  re¬ 
duced  from  the  five  above  to  three  as  follows :  it  would  clearly  be 

unreasonable  to  allow  backorders  for  type  i  units  if  there  were  serviceable 

2 

snare  stock  on  hand;  thus,  N.  -  T  n. .  and  m.  cannot  both  be  tcsitive. 

S  .  j=i  13 

We  thus  set  s.  *  m.  -  [N.  -  l  n... 3,  which  is  the  net  inventory  of  items 
11  1  j=l  1] 

of  type  i  (it  is  negative  if  there  are  backorders  associated  with  type 
i  units).  The  variables  s,  ,s  and  n  ,  then  describe  the  state  of  the 

-l  2.  2x 

system. 

Clearly  all  reasonable  strategies  will  use  items  of  type  i  to  satisfy 
demands  for  units  of  type  i  whenever  these  items  are  available.  Thus  the 
only  question  which  needs  to  be  considered  is  under  what  conditions  to  allow 
the  use  (i.e.,  substitution)  of  a  type  2  item  in  a  type  1  unit.  Moreover, 
in  this  situation  there  always  is  an  optimal  stationary  Markov  policy  (see 
Ross  [2]);  and  hence  it  suffices  to  identify  the  set  of  states  (called  S, 
or  the  substituting  states)  in  which  a  type  2  item  would  be  installed  in  a 
type  1  unit.  Clearly 

s  c  {(*1,s2,n21):  sL  <  0,  s2  >  0} 

The  set  of  states  corresponding  to  the  policy  which  minimizes  the  ex¬ 
pected  number  of  backorders  can  be  found  by  solving  a  linear  programming 
problem  in  which  the  variables  are  non-substituting  states:  one  corresponding 
to  each  possible  state  of  the  system,  plus  substituting  states:  one  corre¬ 
sponding  to  each  state  from  which  a  substitution  could  be  made  (s  <  0  and 
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s„  >0).  In  the  former,  no  substitution  is  made;  in  the  latter,  a  sub¬ 
stitution  is  assumed.  There  is  a  constraint  corresponding  to  each  non¬ 
substituting  state  and  the  objective  function  is  the  expected  number  of 
backorders  corresponding  to  the  states  in  the  solution.  The  basic  variables 
in  the  final  solution  are  those  states  in  the  optimal  policy.  See  [1],  Heath, 
Muckstadt,  and  Shilepsky  for  further  discussion  of  the  linear  programming 
formulation. 

A  program  was  written  to  generate  the  input  for  a  linear  programming 
package,  MPSX.  The  output  was  then  submitted  to  MPSX.  We  present  the  re¬ 
sults  from  two  problems  in  Example  1  and  Example  2 .  Note  that  since  at  each 
state,  an  optimal  policy  merely  specifies  "substitute"  or  "don’t  substitute" , 
it  suffices  to  graph  the  set  of  states  at  which  one  would  perform  a  sub¬ 
stitution. 
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III .  Providing  an  Input  Basis  for  the  Linear  Programming  Solution 

A  modification  of  the  linear  programming  method  for  finding  the 
optimal  solution  was  made  to  allow  specification  of  a  starting  basis 
for  MPSX.  Most  of  the  computational  time  for  determining  the  optimal 
policy  appeared  to  be  used  finding  an  initial  feasible  solution.  Any 
policy  will  give  a  feasible  solution  and  one  possible  policy  is  to 
allow  no  substitution.  Hence  the  basis  of  vectors  which  contains  only 
non-substituting  states  should  correspond  to  a  feasible  solution. 

The  program  which  generated  input  for  MPSX  was  expanded  to  produce 
a  second  file,  specifying  the  variables  corresponding  to  each  of  the 
non-substituting  states  as  a  basic  variable  in  the  starting  solution. 

In  a  small  problem  the  number  of  iterations  was  reduced  by  this  technique 
from  53  to  5.  Larger  problems  which  had  terminated  because  of  accumulated 
roundoff  errors  were  successfully  solved. 

Larger  problems  such  as  that  of  Example  2  were  difficult  to  solve 
even  with  the  starting  basis.  For  such  problems  a  further  refinement  was 
successfully  used.  An  approximation  of  the  optimal  solution  was  obtained 
by  solving  a  smaller  problem  and  the  scaling  to  be  discussed  in  later  work. 
The  approximation  was  then  used  as  a  starting  basis  for  MPSX.  This  re¬ 
duced  the  number  of  iterations  and  again  allowed  us  to  solve  problems  for 
which  the  linear  programming  algorithm  had  terminated  because  of  numerical 
difficulties. 


m'  ™ 


Thus  as  the  values  of  N.  and  M.  grow,  the  number  of  states  for  the 

i  i 

control  problems  grows  very  rapidly.  We  therefore  considered  two  methods 
for  reducing  the  number  of  states  allowed: 

1.  Restriction  of  the  set  of  policies  considered. 

2.  Removal  of  a  class  of  "very  unlikely"  states. 

Such  restrictions  decrease  the  number  of  variables  for  the  linear 
programming  problem  and  hence  increase  the  size  of  the  problems  we  can 
consider.  These  reductions  also  lead  to  greater  efficiency  in  generating 
the  input  for  MPSX. 

1.  Restriction  of  the  set  of  policies:  One  component  of  the  state, 
namely  n2l’  unc*er  our  direct  control.  If  we  choose  to,  we  can  control 
the  process  in  such  a  way  that  never  gets  very  large.  If  we  restrict 

consideration  to  those  policies  which  maintain  n  ^  ^  n^  we  can  re¬ 
duce  the  states  allowed  to  those  having  n^^  £  n0^  The  solution  then 

found  is  still  an  optimal  solution  but  to  a  smaller  problem,  one  in  which 
certain  types  of  substitutions  are  prohibited.  A  reasonable  but  possibly 
not  optimal  solution  for  the  unrestricted  problem  can  then  be  obtained 
for  each  s:  by  ar.  extrapolation  of  the  restricted  solution.  A  typical 

restriction  is  n„,  <  2  or  n„,  <  3. 

21  -  21  - 
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A  comparison  of  solutions  for  restricted  and  unrestricted  state  spaces 
suggests  that  very  little  accuracy  is  lost  by  the  restriction.  This  is 
partially  due  to  the  fact  that  states  with  large  n^  are  reached  so  in¬ 
frequently  that  their  omission  changes  the  behavior  of  the  system  very 
little . 

Example  1  in  Section  II  with  =  4,  =  2,  =  4,  M  =  4  was 

set  up  and  solved  using  only  states  with  n^^  £  2.  The  number  of  states 
was  reduced  from  522  to  361  and  the  solution  shown  in  Figure  3  was  obtained. 

The  restriction  n0^  <_  2  in  Example  2,  Section  II  with  N  =  6, 

=  3,  =  6 ,  =  5  reduced  the  number  of  states  from  791  to  421  and 

gave  the  solution  shown  in  Figure  4. 

Note  in  Figures  2  and  3  that  because  n^  £  2,  no  state  with  n^  >  1 
will  be  in  the  set  of  substituting  states,  but  that  the  results  obtained 
for  n21  =  0  or  1  agree  exactly  with  those  obtained  for  the  full  space. 

We  can  use  these  results  to  guess  (by,  say,  linear  extrapolation  to  larger 
values  of  n21)  good  policies  for  the  original  problem,  or,  alternatively, 
we  can  carry  out  the  ccmputation  with  n^  3  using  the  results  obtained 
to  choose  a  good  initial  basis. 

2.  Removal  of  a  class  of  "very  unlikely"  states:  Although  the  number 
of  possible  states  grows  very  rapidly,  the  number  of  states  which  "can 
reasonably  occur"  grows  much  more  slowly.  For  example,  if  no  substitution 
is  allowed,  standard  approximations  show  that  the  number  of  type  1  items 


.  .  Vu 

m  repair  is  approximately  normally  distributed  with  mean  - —  and  variance 


‘Yn 


n 

(The  corresponding  result  holds  for  type  2  items.)  If  we  let 


r^  (r^)  be  the  number  of  type  1  (2)  items  in  repair,  we  see  that 


T  = 


is  approximately  a  x  random  variable  with  2  degrees  of  freedom. 

It  can  be  seen  (most  easily  by  considering  scaling  results  to  be  pre¬ 
sented  in  a  later  work)  that  allowing  some  substitution  does  not  change 
the  distribution  of  T  very  much. 

We  thus  use  T  to  identify  a  class  of  states  to  be  eliminated. . .we 
rule  out  all  states  for  which  T  is  greater  than  some  tolerance  limit  TCL. 
In  doing  this  we  do  not  allow  those  transitions  which  would  lead  to  an 
eliminated  state.  All  other  transition  probabilities  remain  the  same. 

In  Example  1,  for  example,  eliminating  those  states  for  which  T  >  3 
reduces  the  number  of  states  frcm  522  to  237.  The  policy  obtained  remains 
exactly  the  same.  If  all  states  having  T  >  2.5  are  eliminated,  the  num¬ 
ber  of  states  is  further  reduced  to  211.  The  results  for  this  case  are 
shown  in  Figure  5. 

since  some  states  were  eliminated,  the  policy  is  not  efined  at  some 
of  the  original  states.  Where  it  is  defined  it  agrees  with  the  previous- 

i 

ly  obtained  policy  except  at  the  point  indicated  by  G  in  Figure  5. 

I  As  a  further  illustration.  Example  2  with  all  states  having  T  >  3 

'  I 

•I  eliminated  yielded  the  policy  shown  in  Figure  6. 

i 

'I  In  this  case  the  number  of  states  was  reduced  from  791  to  ^96.  Again 

the  solution  agrees  with  the  exact  solution  for  those  states  not  eliminated. 

A  reasonable  approach  to  restricting  the  statespace  seems  to  be  a  com¬ 
bination  of  the  two  techniques  discussed.  The  following  results  obtained 


J 
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for  Example  1  with  a?1  _j_  2  and  TOL  3  and  only  ISO  states  agreed  with 
the  exact  results.  A  more  useful  result  is  obtained  here  with  only  130 
states  then  in  Figure  3  with  211  states  indicating  that  judicious  combin 
the  two  techniques  is  appropriate. 

Example  2  in  Section  II  with  =  6  =  3  =  6  =  6  was 

run  with  ^  2  and  TOL  3.  Here  the  number  of  states  was  reduced 

from  "’SI  to  332  obtaining,  again,  results  that  agree  with  the  exact  solu 
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V .  Summary  and  Comments  Concerning  Future  Work 

We  have  now  improved  our  ability  to  find  optimal  or  near-optimal  solu¬ 
tions  using  linear  programming  on  small  examples .  3y  providing  a  starting 
basis  for  MPSX  and  by  eliminating  states  from  those  we  consider  we  have 
reduced  the  size  of  the  problem  and  the  amount  of  time  MPSX  takes  to  solve 
it.  This  has  been  helpful  in  reducing  the  cost  of  each  solution  and  in 
lessening  the  effect  of  roundoff  errors.  We  have  been  able  to  obtain  op¬ 
timal  or  near-optimal  solutions  for  many  cases  which  had  been  previously 
unsolvable . 

The  effects  of  this  have  been  two-fold.  First,  we  have  attained  greate 
insight  into  the  structure  of  the  set  of  substituting  states  in  an  optimal 
solution.  For  example,  the  monotonicity  properties  proposed  in  [1]  have 
held  in  each  case  we  investigated.  Second,  we  now  have  the  ability  to 
solve  sufficiently  large  problems  that  the  scaling  technique  to  be  discussed 
in  future  work  will  be  an  effective  way  to  solve  problems  of  any  size. 

We  plan  to  extend  this  approach  to  finding  optimal  policies  for  finite 
state  Markovian  decision  problems . 
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1 .  Introduction 


In  [1]  Heath,  Muckstadt  and  Shilepsky  consider  an  inventory  model 
for  recoverable  items  in  which  one  item  may  be  substituted  for  another 
during  repair.  The  model  is  formulated  as  a  Markovian  decision  problem 
for  which  there  exists  an  optimal  stationary  control  policy.  The 
optimal  policy  can  be  found  by  linear  programming,  but  the  size  of 
most  real  problems  makes  this  approach  impractical.  Its  application 
to  small  problems  led  to  conjectured  properties  that  an  optimal  policy 
should  satisfy.  Based  on  these  properties,  an  efficient  search  method 
was  developed  to  find  good,  but  not  necessarily  optimal  policies. 

The  search  involves  a  comparison  of  policies  by  analytic  methods  or 
simulation.  This  paper  focuses  on  policy  comparison  by  simulation  and 
several  variance  reducing  techniques  that  contribute  to  the  efficiency 
of  the  simulation. 

II.  Formulation  of  the  Model 

We  consider  the  following  model.  We  suppose  that  there  are  two 
types  of  items  in  the  system:  type  1  and  type  2.  The  assemblies  in  which 
these  items  are  installed  are  called  units ;  we  also  assume  that  there 
are  two  types  of  units  and  each  unit  contains  only  one  item  of  the  types 
considered.  All  items  are  assumed  to  be  stored  at  a  single  location. 

We  assume  that  items  of  type  2  can  be  placed  in  units  of  type  1 
or  type  2,  but  that  items  of  type  1  C3n  be  placed  only  in  units  of 
type  1.  Finally,  we  assume  that  times  to  failure  and  repair  times  are 
exponentially  distributed. 
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For  the  model  we  suppose  that : 

1)  There  are  N.  units  of  tyoe  i  and  N.  +  M.  items  of  tyoe 

i  li 

are  available  (i.e.,  there  are  spare  items  of  type  i), 

2)  The  state  of  the  system  at  any  instant  can  be  specified  by  the 
five  numbers : 


n^. ,  the  number  of  type  i  items  installed  in  type  j 
■units ,  i  >_  j  . 

m. ,  the  number  of  serviceable  type  i  units  in  spare 
stock. 


The  number  of  type  i  items  in  repair  is  then  given  by 
i 

N.  +  M.  -  (m.  +  Y  n..)  and  the  number  of  back-orders  associated  with 
i  i  i  il  2 

type  j  units  is  N .  -  £  n . . .  We  suppose  that  the  failure  times  of 

3  i=j 

type  i  installed  in  units  of  type  j  are  independent  exponential 
random  variables  with  mean  1/A^ ,  and  that  repair  times  are  inde¬ 
pendent,  exponential  random  variables  with  mean  1/r^.  The  measure  of 
performance  of  the  system  under  any  substitution  policy  is  the  expected 
number  of  backorders  when  the  system  is  in  equilibrium. 

Under  any  reasonable  control  policy  Ci.e.,  rule  specifying  which 
items  to  use  to  repair  various  units  under  ail  possible  circumstances) 
the  number  of  variables  necessary  to  describe  the  state  of  the  system 
can  be  reduced  from  the  five  above  to  three  as  follows:  it  would 

clearly  be  unreasonable  to  allow  backorders  for  type  i  units  if  there 

2 

were  serviceable  SDare  stock  on  hand:  thus,  N.  -  Y  n. .  and  m. 

1  j=i  ^  1 

2 

cannot  both  be  positive.  We  thus  set  s.  =  m.  -  [M.  -  Y  n..],  which 

i  i  i  13 

J 

is  the  net  inventory  of  items  of  type  i  (it  is  negative  if  there  are 
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backorders  associated  with  type  i  units).  The  variables  s^,  and 
n01  then  describe  the  state  of  the  system. 

Clearly  all  reasonable  strategies  will  use  items  of  type  i  to 
satisfy  demands  for  units  of  type  i  whenever  these  items  are 
available.  Thus  the  only  question  which  needs  to  be  considered  is 
under  what  conditions  to  allow  the  use  (i.e.,  substitution)  of  a  type  2 
item  in  a  type  1  unit.  Moreover,  in  this  situation  there  always  is 
an  optimal  stationary  Markov  policy  (see  [1]);  and  hence  it 
suffices  to  identify  the  set  of  states  (called  S,  or  the  substituting 
states)  in  which  a  type  2  item  would  be  installed  in  a  type  1  unit. 

The  set  of  states  corresponding  to  the  policy  which  minimizes 
the  expected  number  of  backorders  can  be  found  by  solving  a  linear 
programming  problem.  Since  at  each  state  an  optimal  policy  merely 
specifies  ’’substitute'’  or  "don't  substitute",  it  suffices  to  graph 
the  set  of  states  at  which  one  would  perform  a  substitution.  We 
present  the  solution  for  a  typical  problem  in  Figure  1. 

The  most  serious  drawback  to  the  linear  programming  solutions  is 
related  to  the  number  of  states  in  the  Markov  chain  and  hence  the 
number  of  constraints  in  the  linear  programming  formulation.  See  [1], 
p,  10  for  discussion.  We  therefore  investigated  alternative  approaches. 

III.  Search  for  Optimal  Policies 

To  find  a  good  approximation  to  the  optimal  policy  we  employ  a 
search  technique  which  begins  with  the  set  of  substituting  states,  S, 
empty  and  repeatedly  considers  adding  states  one  at  a  time  to  S  by 


comparing  the  performance  of  the  system  using  the  augmented  set  of 
substituting  states  with  the  performance  using  the  current  policy. 

We  developed  two  methods  for  policy  comparison:  one,  analytic  which 
gives  an  exact  number  for  the  expected  backorders  under  a  given  policy, 
and  the  second,  simulation.  Similar  results  obtained  by  linear  program¬ 
ming  and  the  search  with  analytic  policy  comparison  suggest  that  the 
search  mechanism  is  valid.  However,  the  search  with  analytic  policy 
comparison  requires  roughly  the  same  amount  of  computer  time  and 
storage  as  solution  by  linear  programming.  Hence  simulation  was 
investigated  for  more  efficient  policy  comparison.  In  each  case 
examined,  the  results  obtained  by  analytic  and  simulation  comparison 
were  the  same.  See  [1,  p.  10]  for  a  discussion  of  the  search  technique 
and  comparative  results. 

IV.  Policy  Comparison  by  Simulation  and  the  Use  of  Variance  Reducing 

Techniques 

The  simulation  method  performs  two  simulations,  one  for  each  policy 
and  uses  the  state  at  which  the  policies  differ  as  a  starting  point  for 
the  simulation.  In  one,  the  substitution  of  a  type  2  part  is  made,  and 
in  the  other  it  is  not  made.  The  two  simulations  are  then  run  until 
they  both  reach  the  same  state .  This  state  is  not  necessarily  the  one 
in  which  the  simulations  started.  If  either  system  reaches  the  initial 
state  in  which  the  desirability  of  substitution  is  being  questioned,  the 


substitution  is  not  made . 
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When  the  two  systems  reach  the  same  state  the  run  is  terminated. 

This  is  repeated  a  large  number  of  times,  and  the  difference  in 
backorder  days  for  the  system  with  substitution  and  the  system 

without  substitution  is  recorded.  We  compute  the  sample  mean  for 

-  2 

the  difference  m  backorder  days,  X,  and  sample  variance,  o  .  We 
wish  to  determine  E(X)  since  if  E(X)  <  0,  the  substitution  should 
be  made  and  if  E(X)  >0,  it  should  not.  A  confidence  interval  is 
constructed  at  a  selected  confidence  level;  if  this  confidence  interval 
does  not  include  the  origin,  the  procedure  is  terminated  and  the 
appropriate  strategy  selected  as  optimal.  If  the  confidence  interval 
does  include  the  origin,  the  results  of  another  large  group  of  runs 
are  included  with  the  previous  results .  The  procedure  is  repeated 
until  the  confidence  interval  does  not  include  the  origin  or  is 
shorter  than  some  pre-selected  tolerance  level.  In  the  latter  case 
the  decision  to  substitute  or  not  is  based  on  the  sign  of  the  sample 
mean,  but  no  level  of  confidence  assured.  In  these  cases  both  policies 
give  nearly  the  same  performance  level  and  hence  an  incorrect  choice 
would  have  little  effect  on  system  behavior. 

The  number  of  runs  in  each  group  is  large  enough  so  that  the 
central  limit  theorem  may  be  invoked  and  confidence  intervals  constructed. 
It  should  be  noted  that  we  are  conducting  a  sequential  test,  but  in  the 
construction  of  confidence  intervals,  applying  non-sequential  techniques. 
In  practice,  however,  if  the  number  of  runs  in  the  first  group  is  large 
enough,  the  results  are  definitive  and  3  second  group  unnecessary. 

A  major  contribution  to  variance  reduction  of  X  comes  through  the 
parallel  simulation  of  the  systems  with  and  without  substitution.  We 
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note  that  if  X  =  X11  -  Xs  where  X“  and  Xs  are  sample  means  for 
systems  without  and  with  substitution  then 

Var(X)  =  VarCX11  -Xs)  =  Var(j?)  +  Var(3(s)  -  2  CovCX11,^). 

Failure  and  repair  times  for  parts  which  are  common  to  each  system 
are  the  same,  inducing  a  high  correlation  between  Xn  and  Xs. 

The  search  with  policy  comparison  by  simulation  was  run  for  the 
problem  in  Figure  1  keeping  track  of  VarCX11),  VarCX3)  and 
CovCX  ,X  )  after  each  decision.  We  present  these  results  in  Figure  2, 
also  including  the  variance  if  the  systems  were  uncorrelated, 

Var (X11)  +  VarfX3),  and  the  variance  with  the  induced  correlation, 
VarCX11)  +  VarCX3)  -  2  CovCr^X3)  in  the  last  two  columns. 

We  note  that  1C1  and  Xs  are  highly  correlated  and  that  the 
average  ratio  of  correlated  to  uncorrelated  variances  is  .1358, 
hence  we  have  decreased  the  number  of  runs  necessary  to  obtain  the 
same  level  of  confidence  by  a  factor  of  approximately  8. 


7ar(  X11 ) 

VarCX53 ) 

cover1  .Xs) 

Variance  if 
uncorrelated 

Variance  with 
correlation 

68.98 

59.5 

53.17 

128.48 

22.14 

37.06 

55.61 

58.56 

142.67 

25.55 

44.25 

36.13 

31.47 

80.38 

17.44 

24.95 

34.38 

25.18 

59.33 

8.97 

25,74 

17.06 

18.67 

42.3 

5.46 

29.17 

38.59 

29.63 

67.76 

3.5 

4U.58 

32.3 

35.36 

76.38 

5.16 

38.36 

44.08 

37.65 

32.94 

7.64 

47.06 

67.51 

52.11 

114.57 

10.35 

Average  ratio  of  column  5  to  column  4:  .1358 
Average  correlation  coefficient:  .3746 

Figure  2 
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Two  other  factors  contribute  to  shortening  the  run  time  necessary 
to  compare  policies.  First,  each  simulation  is  begun  at  the  state 
where  a  substitution  is  made  in  one  system  and  not  in  the  other  and 
continued  only  until  they  reach  the  same  state.  In  fact  the  results 
obtained  could  have  been  achieved  by  simulating  the  entire  process 
for  the  two  policies.  If  we  were  to  do  this,  there  would  be  no 
difference  in  backorder  days  until  the  systems  reach  the  state  where 
the  substitution  is  performed  or  not  and  the  differences  would  cease 
when  the  systems  reach  the  same  state.  The  simulation,  then,  can  be 
regarded  as  a  series  of  disjoint  cycles,  each  beginning  at  the  state 
in  question.  It  suffices  to  simulate  only  these  cycles. 

Second,  we  make  only  one  substitution  of  the  kind  in  question 
for  each  run,  but  the  policy  we  are  evaluating  requires  a  substitution 
each  time  the  new  state  is  reached.  We  justify  this  by  appealing  to 
the  linear  programming  formulation  and  solution  of  the  problem.  In 
practice  the  strategies  for  an  optimal  solution  are  pure, but  for 
linear  programming,  the  feasible  solutions  are  mixed  strategies 
on  the  states.  In  the  algorithm- an  entering  variable  is  selected  by 
comparing  the  rate  of  change  of  the  objective  function  with  respect 
to  a  small  change  in  the  variable.  A  small  change  from  0  in  a  control 
variable  corresponds  in  a  probabilistic  setting  to  making  a  substitution 
with  a  very  small  probability.  To  compare  policies  by  simulation,  if 
this  substitution  were  made  with  the  same  small  probability,  most 


runs  would  consist  of  single  substitutions  at  the  state  in  question 
since,  even  if  this  state  were  reached  before  the  systems  came  back 


_a_ 


together,  the  probability  of  a  second  substitution  would  be  very 
small.  In  the  simulation,  the  effect  of  not  making  a  second  sub¬ 
stitution  tends  to  be  to  bring  the  systems  together  sooner  and 
shorten  the  run  length. 

We  considered  but  did  not  use  variance  reducing  techniques 
such  as  antithetic  variables  and  stratified  sampling.  Our  intuitive 
feelings  were  that  the  large  number  of  events  in  each  run  would  make 
questionable  their  benefits  and  that  the  computational  complexity  would 
not  be  justified. 

V.  Summary  and  Comments  Concerning  Future  Work 

We  have  developed  an  efficient  method  for  comparing  policies  which 
differ  at  just  one  state.  This  method  appears  to  be  computationally 
superior  to  other  methods  we  have  investigated  in  two  respects :  the 
amount  of  computer  time  does  not  grow  as  rapidly  with  increased  system 
sice  and,  perhaps  even  mere  relevant  in  terms  of  absolute  limitations, 
the  simulations  do  not  use  the  large  amounts  of  storage  required  to 
solve  linear  programming  problems  or  systems  of  equations . 

Continued  investigation  of  the  structure  of  optimal  policies  has 
reduced  the  number  of  states  necessary  to  consider  for  inclusion  in  an 
optimal  policy  [2],  The  development  of  a  scaling  technique  allows 
large  problems  to  be  approximated  by  small  problems  that  can  be  solved 
explicitly  and  the  extrapolated  solution  used  as  a  first  approximation 
for  a  search  with  simulation. 

We  are  currently  investigating  other  models  for  the  interchange- 
ability/substitutability  problem  and  are  able  to  extend  many  of  the 
techniques  developed  for  this  model  to  multi-echelon  and  multi-it 


:em  problems  [  3 
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Abstract 


In  this  paper  we  examine  the  substitutability  problem  for  several 
recoverable  items  that  fail  at  a  single  location.  We  assume  the 
failure  processes  for  each  type  of  item  are  independent,  stationary, 
Poisson  processes.  We  also  assume  the  repair  times  are  independent 
and  exponentially  distributed.  Furthermore,  we  assume  the  system  is 
a  closed  system;  that  is,  no  items  are  added  to  or  deleted  from  the 
system.  We  develop  a  procedure  for  obtaining  an  approximation  to 
an  optimal  substitution  policy  for  this  system.  The  procedure 
uses  a  sequence  of  two-item  inventory  systems  to  model  the  multiple- 
item  system.  We  then  examine  several  properties  that  we  expect 
an  optimal  substitution  policy  to  possess,  and  show  through  an 
example  how  to  use  these  properties  to  describe  a  comprehensive 
optimal  substitution  policy.  Finally  we  identify  some  areas  suggested 
by  this  problem  which  merit  further  investigation. 
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I.  The?  Multiple-Item  Single-Location  Inventory  System 

In  this  section  we  present  a  single  location  inventory  system 
with  several  types  of  items:  type  1  through  type  n.  The  assemblies 
in  which  these  items  are  installed  are  called  units ;  each  unit 
contains  only  one  item.  We  assume  that  the  units  themselves  are  also 
classified  as  being  of  one  of  the  types  from  type  1  to  type  n.  When 
a  unit  of  type  k  fails,  it  can  be  repaired  with  any  serviceable  item 
of  type  k  or  greater.  Thus  there  is  a  one-directional  substitutability 
among  the  items;  an  item  of  type  k  may  be  used  whenever  an  item  of  a 
lesser  type  is  used. 

For  example,  there  may  be  three  units  corresponding  to  successi  ;e 
generations  of  a  computer.  The  three  items  might  be  the  successive 
generations  of  a  circuit  board  installed  as  original  equipment  on  the 
three  generations  of  the  computer.  Each  version  of  the  circuit  board  is 
designed  to  be  used  in  the  current  generation  of  the  computer  and  in  all 
previous  generations,  but  is  incompatible  with  the  newer  generations  of 
the  computer.  In  this  example,  a  second-generation  circuit  board  may  be 
used  in  both  the  first  and  second-generation  computers,  and  a  second- 
generation  computer  may  be  repaired  with  either  a  second  or  third-generation 
circuit  board. 

Let  be  the  number  of  units  of  type  i,  and  let  fi.  be  the 

number  of  spare  items  of  type  i.  Then  the  total  number  of  items  of  type  i 
is  (!J.  +  M. ) .  Let  r...  be  the  number  of  type  i  items  installed  in 
type  j  units,  and  let  m.  be  the  number  of  serviceable  type  i  items 
in  spare  stock . 

According  to  the  substitution  rules  we  have  established,  n..  is 

i] 

sero  whenever  i  is  lens  than  j.  The  number  of  items  of  type  i  in 


reoair  Is  given  by  (N.  +  M.  -  m.  -  J  n..),  and  the  number  of  back- 

i  1  1 

j=1  n 

orders  associated  with  type  i  units  is  (N.  -  }  n..). 

i=] 

We  suppose  that  the  failure  times  of  type  i  items  installed 
in  type  j  units  are  independent,  exponentially  distributed  random 
variables  with  mean  1/X.,..  The  repair  times  of  type  i  items  are 
independent,  exponentially  distributed  random  variables  with  mean  1/y^; 
we  assume  the  repair  time  does  not  depend  on  the  type  of  unit  in  which 
the  item  is  installed  when  it  fails. 

We  suppose  that  no  cannabilization  occurs  in  the  inventory  system, 
so  that  an  item  remains  in  the  unit  in  which  it  is  installed  until  it 
fails.  We  also  suppose  that  no  condemnations  occur,  so  that  every  item 
is  repaired  and  returned  to  service. 

The  measure  of  performance-  of  the  inventory  system  is  the  expected 
number  of  backorders.  It  seem:',  clear  that  an  optimal  policy  would  never 
allow  a  backorder  for  a  unit  of  type  i  when  there  is  a  spare  item  of 
type  i  in  stock.  But  it  is  not  at  all  obvious  whether  to  substitute 
an  item  of  a  type  greater  than  i  or  to  allow  a  backorder  when  there 
are  no  spare  items  of  type  i  in  stock.  This  decision  will  depend  on 
the  substitutability  rules  we  have  described,  on  the  failure  rates  and 
repair  rates,  and  on  the  number  of  items  already  in  use  or  in  repair. 

For  example,  repairing  a  unit  of  type  ]  with  an  item  of  type  i  may 
result  in  subsequent  avoidable  shortages  if  it  happens  that  a  type  j 
item  becomes  available  and  a  type  i  urit  fails.  It  may  be  advantageous 
to  reserve  some  of  the  more  useful  items  so  that  they  will  be  available 
if  they  are  needed  in  a  newer  generation  unit. 
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A  Procedure  for  Obtaining  ,in  Approximation  to  the  Optimal  Substitution 
Policy 

This  section  develops  a  procedure  for  obtaining  an  approximation  to 
the  optimal  substitution  policy  for  the  inventory  system  just  described. 

It  begins  by  recalling  the  solution  for  the  two-item  system  developed  by 
Heath,  Muckstadt,  and  Shilepsky,  and  shows  why  a  new  procedure  is  needed 
to  keep  the  problem  down  to  manageable  proportions.  The  motivation  for 
the  new  procedure  is  given  next,  followed  by  a  stage-by-stage  accounting 
of  how  the  procedure  would  be  implemented. 

To  obtain  an  approximation  to  the  optimal  substitution  policy  for  the 
multiple-item  inventory  system,  we  will  rely  heavily  upon  the  analysis  of 
the  similar  two-item  inventory  system  fcv  Heath,  Muckstadt,  and  Shilepsky. 
They  formulated  the  two-item  inventory  system  as  a  continuous-time  Markovian 
decision  problem  and  showed  how  to  find  the  optimal  stationary  Markovian 
control  policy  by  using  linear  programming.  The  optimal  substitution  policy 
consists  of  a  rule  dictating  whether  to  repair  a  type  1  unit  with  a  type  2 
item  when  no  type  1  item  is  available.  This  rule  depends  on  three  fac¬ 
tors:  the  number  of  backorders  for  type  1  units,  the  number  of  type  2 
items  in  spare  stock,  and  the  number  of  type  2  items  currently  installed 
in  type  1  units. 

Unfortunately,  the  size  of  the  linear  program  and  the  number  of 
factors  needed  by  the  optimal  substitution  rule  grow  rapidly  as  the  number 
of  types  of  items  increases,  and  this  method  of  solution  becomes  impractical 
For  a  three-item  system,  the  optimal  decision  rule  will  depend  on  the  net 
inventory  position  of  each  of  three  items  together  with  the  number  of 
type  2  items  already  installed  in  type  1  units,  the  number  of  type  3  items 
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already  installed  in  type  1  units,  and  the  number  of  type  3  items  already 
installed  in  type  2  units.  For  more  than  three  items,  the  factors  used 
by  the  optimal  decision  rule  are  even  more  numerous.  So  we  must  develop 
a  new  method  or  solution  of  the  multiple-item  problem  for  two  reasons: 
the  linear  program  used  to  solve  the  two-item  problem  becomes  too  large 
to  be  solved  quickly  and  economically ,  and  the  optimal  decision  rule 
becomes  unwieldy  because  it  depends  on  too  many  factors. 

To  motivate  the  approach  we  will  take  in  finding  an  approximate 
solution  to  the  multiple-item  inventory  system,  let  us  consider  the  inven¬ 
tory  system  from  the  perspective  of  the  person  who  manages  the  warehouse 
which  holds  spare  items  of  type  i.  His  job  is  to  receive  serviceable 
spare  items  as  they  are  repaired  and  to  make  available  a  spare  item 
whenever  the  repair  policy  dictates  that  a  repair  should  be  made  with  an 
item  of  type  i.  This  manager  will  be  called  upon  to  make  a  decision 

when  he  holds  some  spare  items  and  an  item  fails  in  a  unit  of  type  j , 

where  j  is  less  than  or  equal  to  i.  His  decision  will  be  either  to 
make  an  item  available  for  use  or  to  hold  his  spare  items  in  reserve. 

We  have  already  mentioned  that  any  reasonable  substitution  policy 
will  always  call  for  the  repair  of  a  type  i  unit  if  a  type  i  item 
is  available.  So  the  only  decision  which  merits  consideration  is 
whether  to  make  available  a  type  i  item  to  a  type  j  unit,  where  j 
is  less  than  i.  When  an  item  is  made  available  to  a  type  j  unit, 
there  is  one  fewer  item  available  to  the  type  i  units.  So  the  inventory 
manager  will  need  to  be  concerned  with  the  duration  of  time  for  which  the 
item  is  on  loan  to  a  type  j  unit  and  the  likelihood  that  he  would  need 

the  item  for  a  type  i  unit  during  the  period  of  the  loan. 
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Here  we  make  a  simplifying  assumption  which  allows  us  to  reduce  the 

number  of  factors  needed  by  an  optimal  decision  rule.  We  will  assume 

that  the  failure  rates  A.,  are  identical  for  all  items  i  in  all 

i] 

units  j,  and  that  the  repair  rates  vk  are  identical  for  all  items  i. 
Thus  the  manager  of  spare  items  of  type  i  will  not  care  what  type  of 
unit  is  in  need  of  repair.  'Whether  it  be  a  type  1  unit  or  a  type  (i-1) 
unit,  one  backorder  will  be  filled  and  one  type  i  item  will  be  unavailable 
for  a  period  encompassing  a  failure  with  rate  A  and  a  repair  with  rate  p. 

This  simplifying  assumption  enables  the  inventory  manager  to  regard 
the  demand  for  spare  items  as  arising  from  only  two  sources:  from  units 
of  type  i  and  from  units  of  type  less  than  i.  This  means  that  we  have 
a  two-item  system:  type  I  is  an  aggregate  comprised  of  all  types  less  than 
i  and  type  II  is  identical  to  type  i.  We  already  know  how  to  find  an 
optimal  decision  rule  for  a  two-item  inventory  system.  We  will  be  able 
to  use  a  sequence  of  two-item  systems  to  find  an  optimal  decision  rule 
for  the  multiple-item  system  if  we  can  adequately  model  the  behavior  of 
the  aggregated  system  comprised  of  all  types  less  than  i.- 

Let  us  focus  our  attention  on  the  aggregated  system.  It  consists  of 
items  and  units  of  types  1  through  (i-1).  Units  fail  and  are  repaired 
with  items  according  to  some  decision  rule.  Sometimes  the  decision  rule 
dictates  that  an  item  not  be  repaired:  sometimes  there  are  no  suitable 
spare  items  available  for  the  repair.  In  these  instances,  a  backorder 
exists  in  the  aggregated  system,  and  a  demand  is  placed  for  a  spare  item 
of  type  i.  The  interaction  between  the  aggregated  system  of  types  1 
through  (i-1)  and  the  system  of  type  i  is  in  these  demands  placed  when 
a  backorder  exists  in  the  aggregated  system,  so  the  essential  feature 
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we  must  capture  in  a  model  of  the  abrogated  system  is  the  distribution 
of  backorders  in  the  system. 

The  steady-state  distribution  of  backorders  in  a  single-item 
system  can  be  computed  analytically.  The  steady-state  distribution 
of  backorders  in  a  two-item  system  can  be  obtained  from  the  two-item 
model  of  Heath,  Muckstadt,  and  Shilepsky,  where  a  linear  program  is 
used  to  compute  the  steady-state  probabilities  of  the  continuous-time 
Markov  chain  model  of  the  system.  We  propose  to  replace  a  two-item 
system  with  the  single-item  system  which  most  closely  matches  the 
backorder  distribution  of  the  two-item  system.  We  will  enumerate  the 
single  item  systems  for  all  reasonably  small  numbers  of  units  N  and 
spare  items  M,  compute  the  mean  and  the  variance  of  the  number  of 
backorders  expected  in  each  system,  and  choose  as  the  best  match  the 
system  which  most  closely  approximates  the  mean  and  the  variance  of 
the  number  of  backorders  expected  in  the  two-item  system. 

With  this  technique  we  can  recursively  model  an  inventory  system 
with  any  number  of  types  of  items.  We  will  begin  by  replacing  types  1 
and  2  with  the  single-item  system  which  most  closely  matches  the 
backorder  distribution  of  the  two-item  system.  We  will  then  consider 
the  resulting  single-item  system  along  with  the  type  3  system  to  be 
another  two-item  system  which  we  will  replace  with  yet  another  single-item 
system.  We  will  continue  in  this  manner  until  we  have  included  all  types 
of  items  in  the  inventory  system. 

A  stage-by-stage  description  of  this  procedure  follows. 


Stage  1 . 


Stage  2. 
(a) 


(b) 


(c) 


Stage  3. 

(a) 

(b) 

(c) 

Stages  4 


Consider  only  types  1  and  2.  Use  the  approach  described  by 
Heath,  Mucks tadt,  and  Shilcpsky  to  find  the  optimal  decision 
rule  for  using  type  2  item:-,  in  type  1  units. 

Add  type  3  to  the  inventory  system. 

Aggregate  types  1  anil  2.  The  inventory  manager  for  type  3 
will  receive  demands  for  spare  parts  from  units  of  types  1 
and  2.  Use  the  results  of  Stage  1  to  determine  the  distribution 
of  total  backorders  in  units  of  types  1  and  2  under  the  optimal 
decision  rule  found  in  Stage  1. 

Find  a  single-item  inventory  system  which  incurs  backorders 
according  to  the  same  distribution  as  that  of  the  total  back¬ 
orders  in  units  of  types  1  and  2.  Denote  this  single-item 
system  as  type  (1,2). 

Use  the  approach  of  Heath,  Muckstadt,  and  Shilepsky  to  find 
the  optimal  decision  rule  for  using  type  3  items  in  type  (1,2) 
units . 

Add  type  4  to  the  inventory  system. 

Use  the  results  of  Stage  2c  to  determine  the  distribution  of 
total  backorders  in  units  of  types  (1,2)  and  3  under  the 
optimal  decision  rule  found  in  Stage  2c. 

Find  a  single-item  inventory  system,  type  (1,2,3),  which 
incurs  backorders  with  the  same  distribution  as  that  of  the 
total  backorders  in  units  of  types  (1,2)  and  3. 

Find  the  optimal  decision  v-ule  for  using  type  4  items  in 
type  (1,2,3)  units. 

through  (n-1).  Continue  in  this  manner  until  type  n  has  been 


added  to  the  inventory  system. 
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The  results  of  this  noquer.ee  of  decision  rules  for  two-item 
systems  can  be  translated  into  an  explicit  rule  for  whether  to  repair 
a  type  j  unit  with  a  type  i  item.  But  this  matter  is  not  entirely 
straightforward.  A  discussion  of  the  considerations  involved  follows 
in  the  next  section. 

3.  The  Form  of  an  Optimal  Substitution  Policy 

This  section  discusses  the  form  of  an  optimal  substitution  policy 
for  the  multiple-item  inventory  system.  It  begins  by  reviewing  the 
structure  present  in  the  optimal  decision  rule  for  a  two-item  system 
and  describes  some  additional  properties  which  an  optimal  decision 
rule  for  a  multiple-item  system  should  possess.  Then  we  show  how  to 
use  these  properties  in  conjunction  with  the  results  of  the  sequence  of 
two-item  inventory  systems  discussed  in  the  previous  section  to  state 
explicitly  the  decision  rule  to  follow  in  the  multiple-item  system. 

Heath,  Muckstadt,  and  Ghilepsky  examined  the  form  of  an  optimal 
policy  for  the  two -item  inventory  system.  They  showed  that  the  states 
of  the  system  could  be  described  by  three  parameters. 

State  =  (s1,s0,n21) 

where  s^  -  net  inventory  of  typo  1  items  ( -N  <_  s^  _<  M^) 
s2  =  net  inventory  of  type  2  items  (-N^  <  <  M^) 

n22  =  number  of  type  2  items  installed  in  type  1  units 

(0  <_  n0^  <_  minO'h.M^}), 


A  decision  rule  specifies  a  eul  set  of  states  from  which  a  type  2  item 
should  be  substituted  it'o  a  type  1  unit.  For  a  decision  rule  to  be 
feasible,  these  states  must  correspond  to  backorders  among  type  1  items 
and  spares  among  type  2  items.  That  is,  the  subset  S  must  satisfy 

S  c  {(s1,s0,n21)  |s  <  0  and  s0  >  0}. 

Every  such  subset  corresponds  to  a  control  policy.  However,  there 
are  three  monotonicity  relationships  that  one  might  expect  the  optimal 
subset  to  have. 

(1)  (s1,s2,n21)  £  SQpt  and  s  <  S;L  implies  (  s  ,s2>n21)  «■:  S^. 

That  is,  as  the  number  of  type  1  backorders  increases,  there 
should  be  a  greater  willingness  to  substitute. 

(2)  ( si ,s2 ,n21>  '  Sopt  and  >  s2  implies  ( si ,s ,n21>  e  SQpt. 

That  is,  as  the  number  of  type  2  spares  increases,  there  should 
be  a  greater  willingness  to  substitute. 

(3)  (si’s2’n21^  £  Sopt  and  n  <  n21  imPlies  ( Si ,S2 ,n>  €  S0pt’ 

That  is,  as  the  number  of  type  2  items  installed  in  type  1 

units  decreases,  there  should  be  a  greater  willingness  to 
substitute.  The  explanation  for  this  is  that  i  smaller  value 
of  n21  means  that  more  of  the  type  2  items  on  loan  are  in 
repair,  and  so  are  closer  to  being  available  again  as  spare  items. 
In  our  procedure  which  models  the  multiple- item  inventory  system 
as  a  sequence  of  two-item  systems,  we  get  a  set  of  rules  specifying 
whether  to  perform  a  repair  or  to  hold  the  spare  items  in  reserve. 
However,  it  is  not  clear  which  repair  is.  to  be  performed.  As  an 
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illustration  of  the  ambiguity  involved  here,  consider  the  stage  in  which 
we  solve  a  two-item  system  where  type  I  is  an  aggregate  of  types  1  and 
2,  and  type  II  represents  type  3.  Suppose  that  the  optimal  policy  calls 
for  substitution  when  (s.  =  -2,  s0  =  1,  and  n?^  =1).  It  seems 
reasonable  to  interpret  (s^  =  -2)  to  mean  that  there  are  two  backorders 
within  the  type  1  and  tyre  2  systems,  (s^  =  1)  to  mean  that  there  is 
one  spare  type  3  item,  and  (n  =  1)  to  mean  that  there  is  one  type  3 
item  installed  in  a  unit  of  types  1  or  1 .  But  given  this,  which  of 
the  two  backorders  are  we  supposed  to  fill  with  the  one  spare  item? 

To  help  us  decide,  we  conjecture  that  the  choice  of  which  repair  to 
make  will  obey  an  additional  set  of  monotonicity  relations. 

(1)  If  a  substitution  is  called  for  and  there  is  a  spire  item  of 

type  i  and  there  are  backorders  of  types  j  and  j  ,  where 

j^<j0<_i,  do  not  fill  the  backorder  of  type  j  .  The  explana¬ 
tion  is  that  this  backorder  would  be  easier  to  fill  later. 

(2)  If  a  substitution  is  called  for  and  there  are  spare  items  of 

types  i^  and  and  a  backorder  of  type  j,  where  j  <_  i^  <  i  , 

do  not  use  the  item  of  type  i0,  The  explanation  is  that  this 
item  is  potentially  more  useful  for  filling  subsequent  backorders. 
The  principle  behind  both  of  these  properties  is  to  fill  any  backorder 
as  inexpensively  as  possible,  saving  as  much  flexibility  as  possible 
for  the  future . 

A  consequence  of  these  monotonicity  properties  is  that  care  must 


be  taken  in  translating  the  optimal  policies  for  the  sequence  of  two- 
item  systems  into  a  comprehensive  policy  for  the  multiple-item  system. 
To  illustrate,  let  us  suppose  that  the  optimal  policy  from  Stage  1 


II 


(where  type  I  is  type  i  and  type  II  is  type  2)  calls  for  no  substi¬ 
tution  when  (3^  =  -1,  =  1,  and  n,^  =  0).  Suppose  also  that 

the  optimal  policy  from  Stage  2  (whore  type  I  is  types  1  and  2  and 
type  II  is  type  3)  calls  for  substitution  when  (s  =  -1,  s^  =  1, 
and  n^^  =  0).  What  should  we  do  if  there  is  a  type  1  backorder, 
a  type  2  spare,  and  a  type  3  spare,  and  there  are  no  items  currently 
on  loan?  The  Stage  1  policy  says  not  to  substitute,  so  there  is  a 
backorder  in  the  aggregated  system  comprised  of  types  1  and  2. 

The  Stage  2  policy  says  to  substitute  a  type  3  item  in  this  set  of 
circumstances.  But  it  would  violate  a  monotonicity  property  if  we 
were  to  fill  a  type  1  backorder  with  a  type  3  item  when  a  type  2 
item  is  available. 

The  resolution  of  this  problem  lies  in  separating  the  question 
of  whether  to  fill  a  backorder  from  the  question  of  which  backorder  to 
fill.  The  optimal  policies  for  the  sequence  of  two-item  models  should 
determine  only  whether  to  fill  some  existing  backorder.  The  mono¬ 
tonicity  properties  should  be  used  to  determine  which  backorder  to 
fill. 

4.  An  Example 

This  section  presents  an  example  of  a  three-item  inventory  system. 
The  steps  taken  in  solving  this  problem  as  a  sequence  of  two-item  systems 
are  outlined.  The  results  are  then  used  together  with  the  monotonicity 
properties  in  formulating  a  policy  which  dictates  whether  to  repair  a 
failed  unit  and  specifies  the  type  of  item  to  be  used  in  the  repair. 


We  begin  with  an  inventory  system  with  units  and  items  of  types 


1,  2  and  3.  Each  unit  requires  the  installation  of  one  working  item 
to  be  in  working  condition.  Type  1  items  may  be  used  only  in  type  1 
units:  type  2  items  may  be  used  in  units  of  types  1  and  2;  type  3 
items  may  be  used  in  any  of  the  units. 

There  are  four  units  of  each  type  and  two  spare  items  of  each  type; 
that  is,  there  are  four  units  and  six  items  of  each  type.  The  times 
until  failure  of  any  item  installed  in  a  unit  are  assumed  tc  be  inde¬ 
pendent,  identically-distributed  exponential  random  variables  having  a 
mean  of  one  time  period.  The  repair  times  are  also  assumed  to  be  inde¬ 
pendent,  identically-distributed  exponential  random  variables  having 
a  mean  of  one  time  period. 

For  the  first  stage,  we  consider  only  types  1  and  2.  Following  the 
procedure  developed  by  Heath,  "uckstidt,  and  S'nilepsky,  an  optimal 
substitution  strategy  for  this  two-item  problem  was  computed  by  the 
mathematical  programming  package  MFSX.  This  strategy  is  presented  below 
in  the  form  of  charts.  Recall  that  we  established  that  the  state  of 
the  two- item  inventory  system  could  be  characterised  by  three  variables: 
the  net  inventory  of  type  1  items,  the  net  inventory  of  type  2  items, 
and  the  number  of  type  2  items  correctly  installed  in  type  1  units. 

A  substitution  policy  can  then  be  characterized  by  a  decision  of  whether 
to  substitute  a  type  2  item  into  a  type  1  unit  for  each  state  where  the 
net  inventory  of  type  1  items  is  negative  and  the  net  inventorv  of 
type  2  i*ems  is  positive.  The  following  charts  depict  those  states 
where  a  substitution  should  be  performed. 


r 


Optimal  substitution  policy  for  types  1  and  2 


Ke* 


-  net  inventory  of  type  1 
items 


s2  -  net  inventory  of  type  2 
items 


n21  -  number  of  type  2  items 

installed  in  type  1  units 


•  -  substitute  a  type  2  item 
into  a  type  1  unit 


O  -  make  no  repair 
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We  can  give  a  verbal  characterisation  of  this  policy.  "Substitute 
a  spare  type  2  item  into  a  broken  type  1  unit  whenever  there  are  two 
available  spare  items  or  whenever  there  are  at  least  two  broken  type  $ 
units  and  no  type  2  items  are  currently  being  used  in  type  1  units." 

Stage  2  calls  for  the  aggregation  of  types  1  and  2  as  a  single 
inventory  system  and  the  introduction  of  the  type  3  system.  We  must 
first  find  a  single-item  system  which  behaves  as  the  combined  type  1 
and  2  system.  To  this  end,  we  use  the  steady-state  probabilities  computed 
by  MPSX  to  determine  the  mean  and  the  variance  of  the  number  of  backorders 
using  the  optimal  policy  just  described  for  the  combined  type  1  and  2 
system.  We  compute 

ExpCbackorders]  =  2.151 
Var [backorders]  =  2.206 

We  next  search  for  a  single-item  inventory  system  which  closely 
matches  this  mean  and  variance.  Fortunately,  the  mean  and  variance  of 
backorders  in  any  single-item  system  can  be  readily  calculated  by 
computer.  To  keep  the  number  of  states  from  becoming  too  large  for 
efficient  computations ,  we  restrict  the  number  of  units  and  the  number 
of  spare  items  to  be  no  more  than  ten.  The  systems  which  best  match  the 
mean  and  variance  of  backorders  are  listed  below. 


Number  of 
Units 

Number  of 
Spare  Items 

Mean  of 
Backorders 

Variance  of 
Backorders 

7 

3 

2.042 

2.163 

8 

4 

2.062 

2.490 

7 

2 

2.513 

2.131 

6 

2 

2.024 

1.812 

8 

3 

2.524 

2.524 

i 
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We  chose  to  use  the  system  with  seven  units  and  three  spare  items. 

We  again  turned  to  MPSX  to  find  an  optimal  strategy  for  the  two-item 
system,  where  type  I  (corresponding  to  types  1  and  2  combined)  had 
seven  units  and  three  spare  items  and  type  II  (corresponding  to  type  3) 
had  four  units  and  two  spare  items.  The  following  charts  depict  the 
optimal  substitution  strategy. 

A  verbal  characterization  of  this  policy  is  to  "substitute  a 
spare  type  II  item  into  a  broken  type  I  unit  except  when  there  is 
only  one  broken  type  I  unit  and  another  type  I  unit  currently  contains 
a  type  II  item."  Under  this  policy  we  expect  the  number  of  backorders 
to  have  a  mean  of  3.031  and  a  variance  of  3.362,  compared  with  a  mean 
of  3.255  and  a  variance  of  3.249  for  the  same  inventory  system  using 
no  substitution. 

We  have  now  solved  the  sequence  of  two-item  systems  and  are  ready 
to  use  the  monotonicity  properties  to  develop  a  comprehensive  strategy 
for  the  three-item  system.  We  first  note  that  there  are  several  sets 
of  circumstances  where  the  results  from  Stage  I  prescribe  no  substitution 
while  the  results  from  Stage  II  prescribe  a  substitution.  One  such 
situation  is  one  type  1  backorder,  one  type  2  spare  part,  and  one 
type  3  spare  part,  with  no  items  on  loan  to  a  unit  of  a  different  type. 

In  such  situations,  we  should  make  the  substitution  but  use* the  mono¬ 
tonicity  properties  and  fill  the  backorder  with  the  type  2  item  instead 
of  with  the  type  3  item. 

There  is  a  very  appealing  rationale  for  interpreting  the  results 
of  the  two  stages  ir.  this  manner.  The  decision  not  to  substitute 
made  in  Stage  I  was  made  without  any  knowledge  of  the  type  3  system. 
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The  decisions  made  at  Stage  I  will  necessarily  be  conservative  because 
the  policy  is  determined  on  tho  basis  of  no  spare  items  being  available 
from  the  type  3  system.  When  the  potential  availability  of  type  3 
spare  items  is  considered,  we  would  expect  to  be  willing  to  substitute 
items  more  freely.  The  type  3  inventory  system  can  be  said  "to  bankroll" 
the  type  2  inventory  system  by  providing  available  type  3  spare  items 
to  fill  the  extra  backorders  the  type  2  system  incurs  as  a  result  of  its 
more  liberal  substitution  policy. 

So  the  optimal  decision  rule  in  this  example  will  be  to  perform  a 
substitution  whenever  it  is  dictated  by  either  the  Stage  1  or  the  Stage  2 
decision  rule.  The  decision  about  which  backorder  to  fill  should  be  made 
in  accordance  with  the  monotonicity  properties,  filling  a  backorder  with 
the  least  versatile  item  able  to  fill  that  backorder. 

5.  Areas  of  Further  Investigation 

This  section  lists  several  areas  of  further  investigation  for 
matters  that  were  either  suggested  or  carefully  avoided  in  the  preceding 
sections . 

(1)  There  is  no  analytical  proof  that  the  preceding  procedure 
will  produce  an  optimal  substitution  policy.  A  simulation  study  could 
be  designed  to  evaluate  the  performance  of  the  procedure  we  suggest 
here  and  compare  it  to  the  performance  of  other  substitution  strategies. 
The  simulation  study  could  also  confirm  that  the  monotonicity  properties 
are  indeed  valid. 


(2)  There  are  variants  of  our  dynamic  programming  approach  which 
merit  some  consideration.  Instead  of  building  a  sequence  of  two-item 


systems  from  the  bottom,  we  could  start  from  the  top  and  work  down, 
matching  the  distribution  of  spare  items  in  the  aggregate  system 
with  a  two-item  system,  instead  of  matching  the  distribution  of 
backorders.  Another  alternative  is  to  combine  at  each  stage  the  pair  of 
adjacent  systems  which  would  be  expected  to  have  the  highest  degree  of 
substitution  (resulting  from  system  i  incurring  many  backorders  and 
system  (i+1)  providing  many  spare  items)  on  the  grounds  that  a  high 
degree  of  substitution  causes  a  pair  of  systems  to  behave  more  as  a 
single  system.  It  would  be  interesting  to  see  if  these  approaches 
produce  similar  optimal  substitution  policies. 

(3)  The  criterion  for  choosing  a  single-item  system  to  replace  an 
aggregate  system  could  be  studied  further.  We  might  be  able  to  produce 
a  closer  fit  to  the  first  two  moments  by  varying  the  repair  rate  along 
with  the  number  of  units  and  the  number  of  spare  items  in  our  search 
for  an  equivalent  single-item  system. 

(4)  The  comprehensive  substitution  policy  may  be  made  more  auto¬ 
mated  instead  of  using  an  ad  hoc  procedure  based  on  the  monotonicity 
properties.  We  illustrate  this  by  continuing  the  example  of  the 
previous  section.  At  Stage  2  an  optimal  policy  was  prescribed  for 
substituting  spare  type  3  items  into  type  1  and  2  units.  The  type 

3  system  could  have  either  zero,  one,  or  two  spare  items.  We  could 
determine  a  comprehensive  substitution  policy  by  returning  to  Stage  1 
to  recalculate  the  optimal  substitution  policy  as  if  the  type  2  system 
had  one  and  two  additional  spare  items.  In  effect,  we  would  be  deter¬ 
mining  an  optimal  policy  by  explicitly  incorporating  the  "bankroll" 
of  spare  type  3  items  into  the  inventory  of  type  2  items . 


(5)  The  monotonicity  properties  alone  do  not  always  enable  us 


to  choose  which  substitution  to  make.  They  do 
tell  us  how  to  choose  between  repairing  a  type 
item  and  repairing  a  type  3  unit  with  a  type  4 


» 

I 
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not,  for  example, 

1  unit  with  a  type  2 
item. 


